neon: route GPT-5 via Responses API + mark image output#3021
Merged
rekram1-node merged 7 commits intoJul 9, 2026
Conversation
Verified every Databricks Foundation Model API endpoint against a live Neon AI Gateway branch (us-east-2). Adds 12 models confirmed working (with live-checked image-input + tool-calling capabilities) and removes gpt-5-5, which the gateway rejects as an unknown model.
The two inline models (no base_model to inherit from) were missing the schema-required `description` field, failing CI validation.
Qwen3.5 122B inherits reasoning=true, so the schema requires reasoning_options. Mirrors the canonical alibaba entry (toggle + budget_tokens).
Merged
3 tasks
Live-verified against the Neon AI Gateway (us-east-2):
- claude-opus-4-8: the gateway rejects `speed:"fast"` +
`anthropic-beta: fast-mode-2026-02-01` with 400 ("speed: Extra inputs
are not permitted" / "invalid beta flag") on both the anthropic and
mlflow routes, so the model does not serve fast mode -> remove
[experimental.modes.fast]. Reasoning is not budget_tokens-based:
`thinking.type:"enabled"` + budget_tokens returns 400 ("use
thinking.type.adaptive and output_config.effort"). The gateway accepts
`thinking.type:"disabled"` (toggle off) and adaptive thinking with
output_config.effort; its own validation error enumerates the accepted
values (low/medium/high/xhigh/max) -> switch to toggle + effort.
- qwen35-122b-a10b: `thinking`, `enable_thinking`, and reasoning budget
fields all return 400 ("unknown field"). The real control is
`reasoning_effort`; the gateway's validation error enumerates
none/low/medium/high, and `reasoning_effort:"none"` returns a plain
(non-reasoning) completion -> switch to effort (none/low/medium/high).
Live-verified against the Neon AI Gateway (us-east-2), same as opus-4-8:
- claude-opus-4-7: fast mode 400s ("speed: Extra inputs are not
permitted"), and `thinking.type:"enabled"` + budget_tokens 400s ("use
thinking.type.adaptive and output_config.effort"). `thinking.type:
"disabled"` (toggle off) and adaptive + output_config.effort both work;
the gateway's own validation error enumerates the accepted effort
values (low/medium/high/xhigh/max). -> remove [experimental.modes.fast],
switch reasoning_options to toggle + effort.
- claude-opus-4-6: fast mode 400s the same way -> remove
[experimental.modes.fast]. Its reasoning_options are left unchanged:
`thinking.type:"enabled"` + budget_tokens is accepted here (returns
thinking blocks), so toggle + budget_tokens is correct for 4-6.
The 12 GPT-5 models are served on Neon's OpenAI Responses route (/ai-gateway/openai/v1), not the mlflow chat-completions default — the codex variants are Responses-only (chat/completions returns 400). Add a per-model [provider] override (shape=responses, openai/v1 api, @ai-sdk/openai) so per-model-aware consumers route correctly, and mark modalities.output with "image" since all 12 support the Responses image_generation built-in tool (verified live). The provider default stays mlflow for the other models.
The gpt-5-4 fast mode advertised `service_tier = "priority"`, but the Neon AI Gateway rejects it on both the Responses and mlflow routes: `400 BAD_REQUEST: service_tier='priority' is not supported by Databricks` (same for `flex` / `auto`; only the implicit `default` tier is accepted). Verified live against the gateway for both gpt-5-4 and gpt-5-4-mini, so the [experimental.modes.fast] block is removed from both.
andrelandgraf
force-pushed
the
neon-gpt5-responses-image
branch
from
8605390 to
fa2474b
Compare
Contributor
|
I have completed my review. Here is my final assessment. ReviewThis PR (stacked on #3019) makes two GPT‑5 corrections to the Neon provider plus a catalog sync. I verified the base-model references, reasoning_options, per-model What checks out:
Finding:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Stacked on #3019 (branched off
neon-catalog-sync-2026-07). Once #3019 merges, this PR's diff reduces to the 12 GPT‑5 model files below.Two related corrections to the Neon provider's GPT‑5 models, both verified live against the gateway:
Route GPT‑5 via the Responses API, not mlflow chat-completions. The Neon provider's default
apiis…/ai-gateway/mlflow/v1, but the GPT‑5 models are served on the OpenAI Responses route (…/ai-gateway/openai/v1). The codex variants are Responses-only — mlflow returns400 "model … is not available on the chat_completions endpoint". So the current entries silently break for consumers that use the provider default. Each of the 12 GPT‑5 models now carries a per-model[provider]override:(Same per-model override pattern Amazon Bedrock uses for
openai.gpt-5.5→ a/openai/v1Responses endpoint.) The provider-level default staysmlflow/v1for the rest of the catalog (Claude/Gemini/Llama/Qwen), which work on chat-completions.Mark
modalities.outputwith"image". All 12 GPT‑5 models generate images via the Responsesimage_generationbuilt-in tool — verified live (each returned a real JPEG, ~33–38 KB). Now that they're pinned to the Responses route where that tool works,output: ["text","image"]is accurate. This also gives the Neon docs an "Image" category via the standardmodalities.output.includes("image")filter (same convention OpenRouter uses for its GPT-image models).Nothing changes for the non-GPT‑5 models.
Models changed (12)
gpt-5,gpt-5-mini,gpt-5-nano,gpt-5-1,gpt-5-2,gpt-5-4,gpt-5-4-mini,gpt-5-4-nano,gpt-5-3-codex,gpt-5-2-codex,gpt-5-1-codex-max,gpt-5-1-codex-miniNotes
modalities.inputis preserved per model (some GPT‑5 addpdf); onlyoutputgainsimage.[provider]overrides varies. models.dev/opencode honor them; some flat-registry consumers (e.g. Mastra's models.dev gateway) currently route by provider id and ignore per-model overrides — I'm opening an upstream Mastra PR to make per-modelshape/apioverrides work generally.Test plan
base_modelmerge validation: all 36 Neon models valid; 12 GPT‑5 resolve tomodalities.output ⊇ [image]andprovider.shape = responses.Update — dropped unsupported fast mode on
gpt-5-4/gpt-5-4-miniBoth carried
[experimental.modes.fast]withprovider.body.service_tier = "priority". The Neon AI Gateway rejects that tier on both the Responses and mlflow routes:service_tierflexandautoare rejected the same way; only the implicitdefaulttier is accepted. Verified live for both models, so the fast-mode block is removed fromgpt-5-4andgpt-5-4-mini.Rebased onto the latest
neon-catalog-sync-2026-07(#3019), so this PR's diff is now exactly the 12 GPT‑5 model files on top of #3019.